Skip to content

p2p: run the EnzinoBB/p2p fork (packet size limit, host data refresh, ordered shutdown) — P9 - #121

Merged
EnzinoBB merged 6 commits into
masterfrom
feat/p2p-fork-p9
Oct 30, 2026
Merged

EnzinoBB merged 6 commits into
masterfrom
feat/p2p-fork-p9

Conversation

@EnzinoBB

@EnzinoBB EnzinoBB commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Plan step P9 of the p2p snapshot work: the node runs its own fork of the p2p library, EnzinoBB/p2p (branch community/p9, commit 09252ec, on top of 8fb3c516, the commit the official node pins). Review: CreditsCommunity/p2p#1. This changes the network transport of every node: not to be merged without the user's review.

p2p changes

  1. Packet size limit. A packet header announcing more than kMaxPacketSize = 256 MiB of data drops the connection with kProtocolCorrupted before any allocation. One LOG(ERROR) line at most once a minute per address, with counts. Packet::Get and Unserializer::Get(std::vector) get the same kind of bound. 256 MiB is far above any packet the node sends; the inbox already drops its second-priority list above 512 MB in total.
  2. Host data refresh. When the sender of a direct datagram is already in its k-bucket, its user_data is updated; address and ports stay as recorded and are checked again under the lock. With full_net_discovery (the node's setting) GetKnownNodes shows the k-bucket value for nodes in the k-buckets, instead of the first value seen, which another node often relayed. Without this the official code kept the first value until the entry was evicted. A received UDP datagram is now passed with its own length, not the whole receive buffer.
  3. Shutdown. ~Host: stop io_, join the io thread, then RoutingTable::Stop() (idempotent: joins the pinger and explorer threads, then really closes the UDP socket; Close() was a no-op) and Network::Instance().SetRoutingTable(nullptr). ~RoutingTable stops the threads before deleting the k-buckets.

Review fixes (p2p 09252ec, node 02a3dd24)

  1. Memory per connection.
    • Before registration a connection accepts only a registration of at most 64 bytes (RegData is 7 or 19 bytes).
    • After registration the limit stays 256 MiB, but data is read in 1 MiB chunks, so the memory held follows the bytes actually received. A header announcing 256 MiB - 1 with no data costs one 1 MiB chunk until the 10 s idle timeout.
    • Why not a lower general cap: the largest messages cannot be bounded tightly from the code.
      • A pool sync reply packs all blocks the requester asked for in one compressed packet; the requester sets the count (blockPoolsCount, 100 by default, configurable).
      • A block is bounded only by consensus parameters: 1000 packets × 1000 transactions × 100 KiB.
      • A 32/64 MiB cap could drop legitimate replies and stall sync with that peer.
    • With chunked reads a large announcement costs the node only what the sender actually transmits; the inbox already bounds queued data at 512 MiB.
  2. Host data refresh. Only a PingResp that answers an outstanding ping of ours (ping_sent_) refreshes an entry; unsolicited datagrams no longer do. A change is seen at the node's next ping, and snapshot-p2p-wait still passes.
  3. Notification threads at shutdown. RoutingTable::Stop waits, at most 5 s, for the detached NotifyHost threads (they call into the host being destroyed); none can start once the io, pinger and explorer threads have ended.
  4. Zero values. A zero k-bucket user_data no longer replaces a non-zero explorer value.

net-huge-frame now checks, with memory read via --pid at each step:

  • over-limit headers before registration (2^40, 65 bytes) and after it (2^40, 256 MiB + 1): closed at once, both limit lines logged;
  • just-below-limit headers without data (64 bytes, 256 MiB - 1): held until the idle timeout.

Node side

  • .gitmodules: p2p from https://github.com/EnzinoBB/p2p.git; the fork's own submodules (leveldb, miniupnp) keep their upstream URLs and commits.
  • ~Transport no longer calls SetRoutingTable(nullptr) (added in Windows and macOS nodes load Linux checkpoints and sync mainnet with the same digests #115): ~Host now does it after its io thread has ended. Clearing it earlier, while the io thread could still read the pointer in Network::OnConnected, was a data race. Clearing twice would have been harmless (a shared_ptr reset); the call is removed so only one safe place remains.
  • CI, node_alt_ref builds (robustness, testnet, scale): the node under comparison (the official node) is built with the p2p commit it points to (8fb3c516). This repository no longer points to the same p2p commit; the alt build checks that the p2p submodules (leveldb, miniupnp) match.
  • New robustness row net-huge-frame (HUGE_FRAME_AT, ci/testnet/huge_frame.py). See above for the cases; every node of this build (the odd ones in mixed runs) must:
    • keep running;
    • grow its memory by no more than 64 MiB at each step;
    • stay in consensus.
  • snapshot-p2p-wait is no longer manual. The capability check now also expects the late serving nodes to be seen as serving.
  • docs/differences.md, docs/CHANGES.md.

Test runs

After the review fixes (02a3dd24):

  • Build Linux + unit tests: 40979065474. Build Windows + macOS: 40979065543. Both success.
  • Robustness, plain: 40979071098. Green: net-huge-frame, garbage-datagrams, net-corrupt, snapshot-p2p-wait, snapshot-p2p-join, snapshot-capability, chaos-mixed.
    • net-huge-frame, n1..n7: over-limit headers closed at once (+0 MiB). The just-below-limit headers were held and then closed on the idle timeout: +4 MiB for 4 connections at 256 MiB - 1, +0 MiB at 64 bytes. Both limit lines were logged and the nodes kept matching digests.
    • snapshot-p2p-wait: the joiner waited for n7; the check reported 1 of 1 late serving nodes do since their restart.
  • Robustness with node_alt_ref=official: 40979079273. Green: net-huge-frame (odd nodes; same figures) and snapshot-p2p-join.

Before the review fixes (7690ffab):

Local, p2p alone with ASan/UBSan:

  • over- and below-limit headers: as above;
  • messages of 0, 1, 1 MiB ± 1, 5 MB and 100 MB delivered intact across chunks;
  • a peer's quick restart with other host data: seen within 10 s through the ping cycle;
  • hosts destroyed without sanitizer reports.

🤖 Generated with Claude Code

EnzinoBB and others added 3 commits October 29, 2026 23:30
… refresh, ordered shutdown)

The p2p submodule now comes from EnzinoBB/p2p, branched from the commit the
official node pins (8fb3c516):
- a packet header announcing more than 256 MiB of data closes the connection
  before any allocation, with one error line a minute per address;
- a peer's host data (version, snapshot capability) is refreshed from its own
  datagrams, and the full-discovery peer list shows that value;
- a received datagram is passed with its own length;
- ~Host stops the routing table (threads joined, UDP socket closed) after its
  io thread and clears the Network singleton's reference to it.

~Transport no longer clears that reference itself: ~Host does it once the io
thread has ended; doing it before ~Host raced with Network::OnConnected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… suite

- run.sh HUGE_FRAME_AT: huge_frame.py opens connections to the p2p TCP port
  of every node of this build and sends packet headers announcing 2^40, 2^30
  and 256 MiB + 1 bytes; each node must close them at once, log the limit,
  keep running with its memory flat and stay in consensus. Row net-huge-frame.
- snapshot-p2p-wait is no longer manual, and the capability check now expects
  the late serving nodes to be seen as serving (peers refresh host data).
- robustness/testnet/scale: the node under comparison (node_alt_ref, the
  official node) is built with the p2p commit it points to, since this
  repository now points to its fork.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EnzinoBB and others added 2 commits October 30, 2026 01:20
…m ping answers

- p2p at 09252ec: a registration may announce at most 64 bytes, later packets
  256 MiB, read in 1 MiB chunks so memory follows the bytes received; host data
  refreshed only from answers to our pings; a zero k-bucket value does not hide
  the explorer's; RoutingTable::Stop waits for the routing-table notification
  threads (at most 5 s).
- net-huge-frame: oversized headers before and after a registration (closed at
  once, both limit lines logged) and just-below-limit headers without data (64
  bytes, 256 MiB - 1: held until the idle timeout), memory checked each step.
- docs: differences.md and CHANGES.md follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@EnzinoBB

Copy link
Copy Markdown
Collaborator Author

Updated with origin/master (a025182, CHANGES rows kept in PR order). CI: Linux run 41068972357 green; Windows/macOS run 41068972310 green; robustness run 41069022155 (net-huge-frame, snapshot-p2p-wait, snapshot-p2p-join) green; official mix run 41069036891 (net-huge-frame, node_alt_ref=official) green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@EnzinoBB
EnzinoBB merged commit 310457a into master Oct 30, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant